MULTIPLICITIES OF THE DISCRETE SERIES 
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Abstract. The purpose of this paper is to show that the multiplicities of a 
discrete series representation relatively to a compact subgroup can be "com- 
puted" geometrically, in the way predicted by the "qantization commutes with 
reduction" principle of Guillcmin-Sternbcrg. 
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1. Introduction and statement of the results 

In the 50 's, Harish Chandra has constructed the holomorphic discrete series 
representations of a real semi-simple Lie group G as quantization of certain rigged 
elliptic orbits, in a way similar to the Borel-Weil Theorem. Here the quantization 
procedure is the one that Kirillov calls geometric in [17j . 

The first purpose of this paper is to explain how one can "compute" geometri- 
cally the mutiplicities of a holomorphic representation of a real simple Lie group 
relatively to a compact connected subgroup : our main result is Theorem 1 1.81 This 
computation follows the line of the orbit method [17j and is a non-compact example 
of the "quantization commutes with reduction" phenomenon [12l [29j [33l [39] . 

Next we show that this result extends to the discrete series representations for 
which the Harish-Chandra and Blattner parameters belong to the same chamber of 
strongly elliptic elements. See Theorem 11.91 

In our previous article |34| , we prove that a similar result occurs : the multiplic- 
ities of any discrete series representation relatively to a maximal compact subgroup 
can be "computed" geometrically. 



Date: November 2008. 

Keywords : moment map, reduction, geometric quantization, discrete series, transvcrsally 
elliptic symbol. 

1991 Mathematics Subject Classification : 58F06, 57S15, 19L47. 

1 



2 



PAUL-EMILE PARADAN 



Nevertheless, our present contribution is not a consequence of the results of [34] . 
for two reasons: 

(1) In |34| . we were working with the metaplectic version of the quantization 
(we prefer the denomination "Spin" quantization). Here we work with the 
geometric version of the quantization. See the review of Vogan [41| for a 
brief explanation concerning this two kinds of quantization. 

(2) The other main difference with [34] is that here we look at the multiplicities 
relatively to any compact connected subgroup, subordinated to the condition 
that the multiplicities arc finite. 

Our main tool to investigate (2) is the "formal geometric quantization" procedure 
that we have studied in [35] . 

Finally, we mention that our present paper is strongly related to the works of 
Kobayashi [21] [221 ESj and Duflo- Vargas [9] where they study the general setting of 
restrictions of irreducible representations to a reductive subgroup. 

1.1. Realisation of the holomorphic discrete series. Let G be a connected 
real simple Lie group with finite center and let K be a maximal compact subgroup. 
We make the choice of a maximal torus T in K . Let g, t, t be the Lie algebras of 
G, K, T. We consider the Cartan decomposition q = t © p. 

Wc assume that G admits holomorphic discrete scries representations. It is the 
case if and only if the real vector space p admits a i-C-invariant complex structure, 
or equivalently, if the center Z(K) of K is equal to the circle group : hence the 
complex structure on p is defined by the adjoint action of an element z a in the Lie 
algebra of Z(K). 

Let A* C t* be the weight lattice : a 6 A* if ia is the differential of a character 
of T. Let d\ C A* be the set of roots for the action of T on g © C. We have 
91 = 9i c U 9l„ where 9i c and 9l n are respectively the set of roots for the action 
of T on i © C and p © C. We fix a system of positive roots 9t+ in 9i c . We have 
p © C = p + © p~ where the /T-module p ± is equal to ker(ad(z ) =F *)• Let 9^' z ° be 
the set of roots for the action of T on p ± . The union 9t+ U 9t+' z ° defines then a 
system of positive roots in 9t that we denote by 9^01- 

Let tl C t* be the Weyl chamber defined by the system of positive roots 9t+. Let 
Choi := <= t+ I > 0, V/3 G 9t+' 2 °}, where (•,•) denotes the scalar product on 

t* induced by the Killing form of g. So the closure Choi is the Weyl chamber defined 
by the system of positive roots 91^- 

The complex vector space p + is an irreducible ^-representation. Hence, if /3 m in 
is the lowest T- weight on p + , every weight (3 € 9t+ ' z ° is of the form f3 = /3 m j n + 
SaGSR + n c* a w hh n a e N. Then we have 

(l.l) C ho i = t;n{£et* | (e,/5 min ) > 0}. 

Note that every £ £ Choi is strongly elliptic: the stabilizer subgroup G^ is compact 
and coincides with the stabilizer subgroup K^. 

For every weight A 6 A* n Choi, we consider the coadjoint orbit 

O k :=G-Ac *. 

For X e g, let VX be the vector field on O a defined by : VX(£) := f t e~ tx ■ £\ t =o, 
£ £ 0\. We have on the coadjoint orbit Oa the following data : 
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(1) The Kirillov-Kostant-Souriau symplectic form J7a which is defined by the 
relation : for any A, Y" G g and £ G Oa we have 

n A (vx,VY)\ s = (n,[x,Y}). 

(2) The inclusion $g : Oa ^ g* is a moment map relative to the Hamiltonian 
action of G on (Oa, ^a). 

(3) A G-invariant complex structure J\ characterized by the following fact. 
The holomorphic tangent bundle T ' Oa — ► Oa is equal, above A € Oa, to 
the T-module 

P~ 

(4) The line bundle Ca '= G Xa' a Ca over G/K A ~ Oa with its canonical 
holomorphic structure. Here Ca is the one dimensional representation of 
the stabilizer subgroup Ka attached to the weight A 6 A*. 

One can check that the complex structure Ja is positive relatively to the symplec- 
tic form, e.g. £Ia(~ , ^A~) defines a Riemannian metric on Oa- Hence (Oa, ^a, J a) 
is a Kahlcr manifold. Moreover the first Chern class of Ca is equal to [^-1 : the 
line bundle Ca is an cquivariant pre- quantum line bundle over (Oa,^a) |25[ 138]. 

We are interested in the geometric quantization of the coadjoint orbits Oa, A G 
A* n Choi- We take on Oa the invariant volume form defined by its symplectic 
structure. The line bundle Ca is equipped with a G-invariant Hcrmitian metric 
(which is unique up to a multiplicative constant). 

Definition 1.1. We denote Qg(Oa) the Hilbert space of square integrable holo- 
morphic sections of the line bundle Ca — > Oa ■ 

The irreducible representations of K are parametrized by the set of dominant 
weights, that we denote 

K := A* m+. 

For any p G K, we denote the irreducible representation of K with highest 
weight p. 

Let p n be half the sum of the elements of £H+' Z °. Let S^p -1 ") be the symmetric 
algebra of the vector space p + : it is an admissible representation of K since the 
center Z(K) acts on p + as the rotation group. 

The following theorem is due to Harish Chandra [13] . See also the nice exposition 
[Si- 
Theorem 1.2. Let A G A* nChoi- Then 

• (A, /3 min ) < 2(p n ,/3 m i n ), the Hilbert space Qg(Oa) "is reduced to {0}. 

• //(A,/3 m i n ) > 2(p n ,/3 m i n ), the Hilbert space Qg(Oa) is an irreducible rep- 
resentation of G such that the subspace of K -finite vectors is isomorphic to 
V A K ^S(p+). 

The holomorphic discrete series representations of G are those of the form 
Q g (Oa), for A G K n Cjf ol where 

(1-2) C h - oI ={£et+ | (e-2 Pn ,Anin)>0}. 
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Here, we have parametrized the holomorphic discrete series representations 
Qg(Ca) by their Blattner parameter A G K l~l C^ ol . The corresponding Harish- 
Chandra parameter is A := A + p c — p n , where p c is half the sum of the elements 
of !SH+ . One checks that the map A i— > A + p c — p n is a one to one map between 
K n C^ ol and 

(1.3) G ho i := {A G t* | (A, a)>0Vae 9^ ol and A + p n + p c G A*}. 

Example 1.3. Let us consider the case of the symplectic group G = Sp(2,K). Here 
K is the unitary group U(2), and the maximal torus is of dimension 2. In the figure 
we draw the chambers C^ ol C Choi C a is the unique positive compact root, 
and /3i,/32,/?3 ire the positive non-compact roots. The root (3^ corresponds to the 
root /9 m in used in U.l]) . 




/ a 

Figure 1. The case of Sp(2,R) 

1.2. Main results concerning the holomorphic discrete series. Let H C K 

be a compact connected Lie group with Lie algebra \). The if-action on (0a, ^a) 
is Hamiltonian with moment map &h '■ 0\ — > ()* equal to the composition of 
<1>G : Ca - * 0* with the projection g* — > f)*. 

Notation 1.4. We denote Qh{Oa) the (dense) vector subspace of Qg(Ca) formed 
by the H-finite vectors. 

When A G C^ oh we know thanks to Theorem 1 1.2[ that Qh{Oa) is the "restric- 
tion" of the /^-representation Vpj ® S{p + ): we will also denote it as ®S{p + )\n. 
We are interested in the case where the 7i-multiplicities in ® S(p + )\h arc finite, 
e.g. ® S(p + )\h is H-admisssible. 
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The asymptotic K -support of a if -representation E is the closed cone of t+, 
denoted by ASk{E), formed by the limits lim n _*oo e„/i„, where (e n )neN is a sequence 
of non- negative real numbers converging to and (/i ra )neN is a sequence of if such 
that how. K {V* n ,E) ^ for all n E N. 

For any closed subgroup H of if, we denote f)- 1 C 6* the orthogonal for the 
duality of the Lie algebra of H . We have the following result of T. Kobayashi. 

Proposition 1.5 ([2lJ El])- Let E be an admissible K -representation. Let H be a 
compact subgroup of K . Then the following two conditions are equivalent: 

(1) E\h is H -admissible. 

(2) AS K (E)nK-t) ± = {0}. 

Let {71, . . - ,7r} be a maximal family of strongly orthogonal roots (see Section 
[5]). Schmid [36] has shown that S'(p + ) is a if- representation without multiplicity, 
and that the representation occurs in >!5(P + ) if and only if 

r 

«fc(7i H 1" 7fc)> witn n fc e N - 

fe=i 

Thus, we check easily that the asymptotic if -support of Vjf ® S{p + ) is equal to 

^ M >o (7i + ... + 7fc) . 
fc=i 

The following proposition is proved in Sections Q] and O (see Theorem I5.6|) . 

Proposition 1.6. Let A E Choi- TTie following statements are equivalent. 

(1) TTie representation <E) S(p + )\h is admissible. 

(2) We have ££=1 R^°( 7l + • • • + lk ) n if • r/ = {0}. 

(3) TTie map : a - * t)* is proper. 

We know from Proposition 11.51 that (1) and (2) are equivalent, thus our main 
contribution is the equivalence with (3). Nevertheless our proof of Proposition II. 61 
does not use directly the result of Proposition 1 1.51 We prove in Section [4] that (1) 
and (3) are both equivalent to the condition 

A K (p)nA'-h ± ={0}, 

where Ak(P) C tl is the Kirwan convex set associated to the Hamiltonian action 
of if on p. In Section [5] a direct computation gives that 

r 

(1.4) A*(p) =^R^°( 7l + ... + 7fe ). 

k=l 

Another way to obtain p.4[) is by using the theorem of Schmid (which computes 
the if-multiplicities in 5'(p + )) together with the following fact: for any afhne variety 
X C C" which is invariant relative to the linear action of if on C™, the Kirwan set 
Ak(X) is equal to the asymptotic if -support of the algebra C[X] of polynomial 
functions on X (see the Appendix by Mumford in [31]). 

Example 1.7. Since the representation Qz(k)(@a) is admissible, the representa- 
tion Qh(Oa) will be admissible for any subgroup H containing Z(K). 
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The irreducible representations of the compact Lie group H are parametrized by 
a set of dominant weights H C f)*. For any /i£ff, we denote V^ 1 the irreducible 
representation of H with highest weight /i. 

We suppose now that the moment map <E># : 0\ — > ()* is proper, and one wants 
to compute the multiplicities of Qh(Oa)- 

If £ G f)* is a regular value of , the Marsden-Wcinstein reduction 

(Oa) c :=<S> H \H-t)lH 

is a compact Kahler orbifold. If moreover £ is integral, e.g. £ = /i G H, there exists a 
holomorphic line orbibundle C(n) that prequantizes the symplectic orbifold (0\) . 
In this situation, one defines the integer 

Q((Oa) m ) ez, 

as the holomorphic Euler characteristic of ((Ca) m , 

In the general case where // is not necessarily a regular value of Q ((0a) m ) G 
Z can still be defined (see [29l [33]). The integer Q ((Oa) ai ) only depends on the 
data (Oa,£a,J\) in a small neighborhood of §J I 1 (fj,) : in particular Q((Oa)/J 
vanishes when /x does not belong to the image of <E>#. 

Now we can state one of our main result. 

Theorem 1.8. Consider a holomorphic discrete series representation Qg{Oa) with 
Blattner parameter A G C^oi- Let H C K be a compact connected Lie group such 
that the representation Qh{Oa) is admissible. Then we have 

Q h (0a)=£Q((Oa),)C- 

In other words, the multiplicity of in the holomorphic discrete series represen- 
tation Qg(Oa) is equal to Q((Ca) A i)- 

A question still remains. When fj, G H is a regular value of the moment map 
Theorem II. 81 savs that the multiplicity itia(^) of the irreducible representation 
in Qh(Oa) is equal to the holomorphic Euler characteristic of line orbibundle 
— > (Oa)^- Does the multiplicity iiia(/i) coincides with the dimension of the 
vector space 

H\{Oa)^C^)) 
of holomorphic sections of C(n) — > (Oa)^i ? 

1.3. Main result concerning the discrete series. We work now with a real 
semi-simple Lie group G such that a maximal torus T in K is a Cartan subgroup 
of G. We know then that G has discrete series representations [14] . Nevertheless, 
we do not assume that G has holomorphic discrete series representations. 

Harish- Chandra parametrizes the discrete series representations of G by a dis- 
crete subset Gd of regular elements of the Weyl chamber t?j_ [14] . He associates to 
any A G Gd an irreducible, square integrable, unitary representation Tlx of G : A 
is the Harish- Chandra parameter of H.\. The corresponding Blattner parameter of 
H\ is 

A(A) := \- Pc + Pn {\) G A*, 
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where p n (X) is half the sum of the non-compact roots (3 satisfying (f3, A) > 0. 
We work under the following condition 

(1.5) (/?, A)(/3, A(A)) > for any /3 G 9V 

The set of strongly elliptic elements of the Weyl chamber t+ decomposes as an 
union C\ U • • • U C r of connected component : each chamber Ci corresponds to a 
choice of positive roots C 9\ containing $K+. Condition (|1.5|) asks that A and 
A(A) belong to the same chamber Ci. 

When G admits holomorphic discrete series, there is a particular chamber Choi of 
strongly elliptic elements such that the intersection Gd H Choi is equal to the subset 
Ghoi defined in (|1.3p . We noticed already that the map A i— > A(A) defines a one to 
one map between Gd H Choi and K n C^ ol . In particular, any A G Gd fl Choi satisfies 
Condition 11.51 We give in Section 16.21 some examples where Condition (|1.5|) does 
not hold. 

Let A G Gd satisfying (| 1 . 5|) . The coadjoint orbit Ca(a) is pre-quantized by the 
line bundle G x k a(x) Ca(a)- One of the main difference with the holomorphic case is 
that the orbit Oa(a) is equipped with an invariant almost complex structure Ja(X) ? 
which is compatible with the symplcctic form, but which is not integrable in general. 

Let H be a compact connected Lie subgroup of K. Suppose that the moment 
map, <S>h '■ Ca(a) — * V corresponding to the Hamiltonian action of H on 0a(A)j is 
proper. The reduced spaces (0a(a))^ := ^h(H ' (j)/H are in general noi Kahler. 
Nevertheless, their geometric quantization Q ((Oa(A))^) G Z are well defined as the 
index of a Dolbcault-Dirac operator (see p9ll33] l. 

The following theorem is proved in Section [6l 

Theorem 1.9. Consider a discrete series representation Tt\ with a Harish- Chandra 
parameter A G Gd satisfying condition \1.5\) . Let H C K be a compact connected 
Lie subgroup such that the moment map $h ■ ^a(a) — * f)* ' s proper. Then 

• the representation H.\\h is admissible, 

• we have 

In other words, the multiplicity of in the discrete series representation 7i\ is 
equal to the quantization of the symplectic reduction (Oa(A))/j- 

Theorem II .91 applies for (most of) the discrete series, but is less precise than the 
results described in Section fQI We expect that: 

(1) The properness of <&h ■ Ca(a) — * f)* should only depend of the chamber Ci 
containing A(A). 

(2) The properness of the the moment map : Ca(A) — * f)* should be equiv- 
alent to the admissibility of the restriction H\\h- 

Duflo- Vargas [S] have shown that the admissibility of the restriction TC\\h is 
equivalent to the properness of the moment map 0\ — > f)*. Since we assume that 
A(A) and A belong to the same chamber, point (1) induces point (2). 

Something which is also lacking is an effective critcrium which tells us when the 
map <£># : Oa(a) - * f)* is proper. See [9] for some results in this direction. 

Acknowledgments. I am grateful to Michel Duflo and Michele Vcrgnc for valu- 
able comments and useful discussions on these topics. 
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2. Quantization commutes with reduction 

In this section, first we recall the "quantization commutes with reduction" phe- 
nomenon of Guillcmin-Sternberg which was first proved by Meinrenken and 
Meinrenken-Sjamaar [28l [29]. Next we explain the functorial properties of the 
"formal geometric quantization" of non-compact Hamiltonian manifolds [35 . 

2.1. Quantization commutes with reduction: the compact case. Let M 

be a compact Hamiltonian if -manifold with symplcctic form f2 and moment map 
$k ■ M — > i* characterized by the relation 

(2.6) L(VX)tl = -d($K,X), let, 

where VX is the vector field on M generated by X € t. 

Let J be a if -invariant almost complex structure on M which is assumed to 
be compatible with the symplectic form : f2(— , J— ) defines a Riemannian metric 
on M. We denote RR (M, — ) the Riemann-Roch character defined by J. Let us 
recall the definition of this map. 

Let E — > M be a complex if -vector bundle. The almost complex structure on 
M gives the decomposition AT*M®C = A' J T*M of the bundle of differential 
forms. Using Hcrmitian structure in the tangent bundle TAI of M, and in the 
fibers of E, we define a Dolbeault-Dirac operator d E + d E : A°' even (M,E) ->• 
A°' odd (M, E), where A l ' j (M, E) := T(M, A^T*M ® c E) is the space of £- valued 
forms of type (i,j). The Riemann-Roch character RR (M,E) is defined as the 
index of the elliptic operator d E + d E : 

RR K (M, E) = Indcx^ (d E + d* E ) 

viewed as an element of R(K), the character ring of K. 

In the Kostant-Souriau framework, a Hamiltonian if-manifold (M, fl, &k) is 
pre-quantized if there is an equivariant Hcrmitian line bundle C with an invariant 
Hcrmitian connection V such that 

(2.7) L(X) - V vx = i{$K,X) and V 2 = -iQ, 

for every let. Here L(X) is the infinitesimal action of X 6 t on the sections 
of £ — > M . (£, V) is also called a Kostant-Souriau line bundle. Remark that 
conditions (|2.7[) imply, via the equivariant Bianchi formula, the relation (|2.6[) . 
We will now recall the notion of geometric quantization. 

Definition 2.1. When (M, f2, $k) is prequantized by a line bundle C, the geometric 
quantization of M is defined as the index RR K (M,£) : we denote it 

Qk(M,CI) g R(K), 

In order to simplify the notation, we will use also the notation Qk(M) for the 
geometric quantization of (M, 51, $k). 

Remark 2.2. Suppose that (M, f2, J) is a compact Kahler manifold pre-quantized 
by a holomorphic line bundle C Then 

• Qk{M, Q) coincides with the holomorphic Euler characteristic of(M,C), 

• for k £ N large enough, Qk(M, fcf2) £ R{K) is equal to the K -module formed 
by the holomorphic sections of C® k — > M . 
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One wants to compute the A'-multiplicities of Qk{M) in geometrical terms. A 
fundamental result of Marsden and Weinstein asserts that if £ G t* is a regular 
value of the moment map $, the reduced space (or symplectic quotient) 

Mi := <S>-k(0/IH 

is an orbifold equipped with a symplectic structure f^. For any dominant weight 
/i G K which is a regular value of 

CQi) := (£|* K _ 1(M) ®C_ M )/A^ 

is a Kostant-Souriau line orbibundle over fi^). The definition of the index of 
the Dolbeault-Dirac operator carries over to the orbifold case, hence Q(M^) G Z 
makes sense as in Definition 12.11 In [29] , this is extended further to the case of 
singular symplectic quotients, using partial (or shift) de-singularization. So the 
integer Q(M /J ) G Z is well defined for every fi € K : in particular <2(M M ) = if 
/j £ *x (M). 

The following theorem was conjectured by Guillemin-Sternberg |12j and is known 
as "quantization commutes with reduction" [28, 29, 39, [33]. For complete references 
on the subject the reader should consult [37l [40] . 

Theorem 2.3 (Mcinrcnken, Meinrenken-Sjamaar). We have the following equality 
in R{K) : 

IJ.GK 

2.2. Formal quantization of non-compact Hamiltonian manifolds. Suppose 
now that M is non- compact but that the moment map $k '■ M — > 6* is assumed 
to be proper (we will simply say "M is proper"). In this situation the geometric 
quantization of M as an index of an elliptic operator is not well defined. Neverthe- 
less the integers Q(M M ),/x G K are well defined since the symplectic quotients M M 
are compact. 

A representation E of K is admissible if it has finite A-multiplicities : 
dim(homif(V^, E)) < oo for every \i G K. Let R~°°(K) be the Grothendieck 
group associated to the A-admissible representations. We have an inclusion map 
R(K) ^ R-°°(K) and R-°°(K) is canonically identify with hom z ( J R(A"), Z). More- 
over the tensor product induces an i?(A)-module structure on R~°°(K) since E®V 
is an admissible representation when V and E are, respectively, a finite dimensional 
and an admissible representation of K. 

Following [55] [35] , we introduce the following 

Definition 2.4. Suppose that (M, f2, $>k ) is proper Hamiltonian K -manifold pre- 
quantized by a line bundle C The formal geometric quantization of (M, f2) is the 
element of R~°° (K) defined by 

When the symplectic structure fl is understood, we will write Q^°°(M) for the 
formal geometric quantization of (M, J7, $^). 
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For a Hamiltonian K -manifold M with proper moment map $>k , the convexity 
Theorem [HI [26] asserts that 

(2.8) Ajr(M) :=$ K (M) nt% 

is a convex rational polyhedron, that one calls the Kirwan polyhedron. 
We will need the following lemma in the next sections. 

Lemma 2.5. Let (M, Q,m) an d (N, Ojy) &e two prequantized proper Hamiltonian 
K-manifold. Suppose that Q'^ x '{M 1 kVL^[) = Q^°°(N, fcOjv) /o?" integer k > 1. 
TTierc Ajf(M) = A K (N). 

Proof. We check that for any /i G A the multiplicity of V fc 7 ^ in Q^°°(M, fcfijw) 
is equal to Q(M M , /cfi^). The Atiyah-Singer Riemann-Roch formula gives us the 
following estimate 

Q(M M fcO M ) ~ est fc r vol(M M ) 

when k goes to infinity. Here est > 0, r — dimM^/2 and vol(M M ) is the symplectic 
volume of M M . Hence, the hypothesis "Q^°°(M, fcOjw) = Qk°° (N, kilpj) for any 
integer fc > 1" implies that $jc(M) n -A = §k{N) n A'. 

Take an integer A > 1 . By considering the multiplicities of in 2^-°° (M,kRQ M ), 

we prove in the same way that $x (M) n j^K = $^(iV) (~l j^-A. Finally we get that 

*jf(M)n{^ \ti£K,R>l} = <S> K (N)n{^ \fi€K,R>l}. 

The proof follows since | /x € A", A > l} is a dense subset of the Weyl chamber 
t*. □ 

Let tp : H — > A be a morphism between compact connected Lie groups. It 
induces a pull-back morphism 95* : R(K) —> R(H). We want to extend ip* to some 
elements of R~°°(K). For /i e A and A G A', let be the multiplicity of Vjj 1 in 
ip*V^ . Formally, the pull-back of E = J2\£K ax ^x < by V i s 

(2.9) ^*A = £ with 6 M = ^ a\N^ . 

Definition 2.6. Let tp : H — > K be a morphism between compact connected Lie 
groups. The element E = J^Aeif a ^\ C * s H -admissible if for every [i E H , the set 
{A G A |oaA^ 7^ 0} is /ira'te. 37ien tfie pull-back ip*E G R~°°{H) is defined by 

The clement y)*A G R-°°(H) is called the "restriction" of A to A, and will be 
sometimes simply denoted by A|#. 

We prove in |35j the following functorial properties of the formal quantization 
process. 

Theorem 2.7. [PI] Let Mi and M 2 be respectively pre-quantized proper Hamil- 
tonian K\ and K 2 -manifolds : the product Mi X M 2 is then a pre-quantized 
proper Hamiltonian K\ x li.2-manifold. We have 

(2-10) QkTxk 2 (Mi x M 2 ) = (Mi)® fi£° (M 2 ) 

in R- 00 ^! x A 2 ) ~ A~ 00 (Ai)§A- 00 (A 2 ). 
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[P2] Let M be a pre- quantized proper Hamiltonian K -manifold. Let if : H — > K 
be a morphism between compact connected Lie groups. Suppose that M is 
still proper as a Hamiltonian H -manifold. Then Q^°°(M) is H -admissible 
and we have the following equality in R~ 00 (if) : 

Qk°°(M)\ h = Q- H °°{M). 

[P3] Let N and M be two pre-quantized Hamiltonian K -manifolds where N is 
compact and M is proper. The product M x N is then proper and we have 
the following equality in R~°°(K) : 

(2.11) Qk°°{M xff) = Q K X (M) ■ Q K (N) 

Property [P2] is the hard point in this theorem. In [35], we have only consider 
the case where if is the inclusion of a subgroup. In Appendix 17.31 we check the 
general general case of a morphism (p : H — > K . 

2.3. Outline of the proof of Theorem II. 8i We come back to the setting of the 
introduction. We consider the holomorphic discrete series representation Qg{Oa) 
attached to the Blattner parameter A e C[~ ol . Recall that the coadjoint orbit Oa ~ 
G/Ka, which is equipped with the Kirillov-Kostant-Souriau symplectic form VIa, 
is pre-quantized by the line bundle C\ := G x^ A Ca- 

Consider first the Hamiltonian action of K on Oa (here if is a maximal compact 
subgroup of G) . One knows that the corresponding moment map $k : Oa — > t* is 
proper |10ll3"!2] . Hence the formal quantization Q]^°{Oa) of the if-action on Oa is 
well-defined. 

Theorem 11.21 tells us that the restriction of the representation Qg(Oa) to K is 

Qk {Oa) = Vf ® S(p + ). 

Theorem 11.81 restricted to the case where H = K, is then equivalent to the 
identity 

(2.12) Qk°°(Oa) =Va* ®S(p+) in R~°°(K), 

that we prove in Section [3] 

Consider now the situation of a closed connected subgroup H of K, such that the 
restriction Qh(Oa) is admissible, e.g. the moment map : Oa ~ > f)* is proper 
(see Proposition II. 6p . We can apply property [P2] of Theorem 12.71 The formal 
quantization 0^(0 a) of the iJ-action on Oa is equal to the restriction of the 
formal quantization Q^-°°(0a) of the if-action on Oa- Hence (|2 . 1 2[) implies that 

Qh°°(Oa) = Qh(O a ). 

So Theorem 11.81 is proved for all the admissible restrictions Qh(Oa), when one 
proves it for the case H = K. 

3. Computation of Q^-°°(0a) 
In this section we prove the following 
Theorem 3.1. Let Oa be the coadjoint orbit passing through A £ Choi- We have 

Q- k °°{Oa) = V«®S{p + ). 
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Similar computation was done in |34j in the setting of a geometric quantization 
of the "Spin" type. 

Note that the formal quantization of 0\ behave differently from the "true" one, 
defined in Definition ll.il when A G Choi \Choi : m this case Qk(Oa) = {0} whereas 

Q7<°°(Oa) + {0}. 

The proof of Theorem 13.11 is conducted as follows. We introduce in Section l3~2l 
a AT-transversaly elliptic symbol a a on 0\. A direct computation, done in Section 
13.31 shows that the AT-cquivariant index of a a is equal to V? ® S^p" 1 "). In Section 
13. 4i we use a deformation argument based on the shifting trick to show that the 
index of a a coincides with Q^ 00 (Oa)- Putting these results together completes the 
proof of Theorem 13.11 

3.1. Transversaly elliptic symbols. Here we give the basic definitions from the 
theory of transversaly elliptic symbols (or operators) defined by Atiyah and Singer 
in [1] . For an axiomatic treatment of the index morphism see Berline-Vergne [6l [7] 
and for a short introduction see [33) . 

Let M be a compact A'-manifold. Let p : TM — ► M be the projection, and let 
( — > ~)m be a AT-invariant Ricmannian metric. If A , A 1 are AT-equivariant complex 
vector bundles over M, a AT-equivariant morphism a G T(TAf, hom(p*A°,p*A 1 )) 
is called a symbol on M. The subset of all (m, v) G TM where a(m, v) : — > E} n 
is not invertible is called the characteristic set of a, and is denoted by Char(cr). 

In the following, the product of a symbol cr by a complex vector bundle F — ► M, 
is the symbol 

a® F 

defined by a ® F(m, v) = cr(m, v) g) IdF m from A^ ® A m to A^ ® A m . Note that 
Char(cr (g A) = Char(cr). 

Let Tk M be the following subset of TM : 

T K M = {(m, w) G TM, (u, VJf (m)) M = for all let}. 

A symbol cr is elliptic if cr is invertible outside a compact subset of TM (Char(cr) 
is compact), and is transversaly elliptic if the restriction of a to TkM is invertible 
outside a compact subset of T^-M (Char(er)nTz<-M is compact). An elliptic symbol 
cr defines an element in the equivariant AT-theory of TM with compact support, 
which is denoted by (TM), and the index of cr is a virtual finite dimensional 
representation of K, that we denote Index^(cr) G R(K) [21 HI [3] - 

Let 

RT C °°{K)CR-°°(K) 
be the A(A')-submodule formed by all the infinite sum Y] g t 71 ^^ where the map 

/j, G AT i— > G Z has at most a polynomial growth. The A(AT)-module R^ C °°(K) is 
the Grothendieck group associated to the trace class virtual ^-representations: we 
can associate to any V G R^ C °°(K), its trace k — > Tr(fc, V) which is a generalized 
function on K invariant by conjugation. In Section 13 - 31 we use the fact that the 
trace defines a morphism of A(AT)-modulc 

(3.13) R^ C °°(K) <—* C~°°(K) K . 

A transversaly elliptic symbol cr defines an clement of Kjf (T^M), and the in- 
dex of cr is defined as a trace class virtual representation of AT, that we still denote 
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Index M (er) £ R^ C °°(K). See [I] for the analytic index and [SIE] for the cohomolog- 
ical one. Remark that any elliptic symbol of TM is transversaly elliptic, hence we 
have a restriction map (TM) — > Kx(T^M), and a commutative diagram 



(3.14) 



Kr-(TM) 



K^(TkM) 



Index! 



Index! 



R(K) 



Using the excision property, one can easily show that the index map Index w : 
Kjf(TjfW) — > R^ C °°(K) is still defined when W is a A-invariant relatively compact 
open subset of a A- manifold (see [33] [section 3.1]). 

3.2. The transversaly elliptic symbol cta- Let A <G Choi- Let us first describe the 
principal symbol of the Dolbeault-Dirac operator dc A +d CA defined on the coadjoint 
orbit Oa- The complex vector bundle (T*0a) 0,1 is G-equivariantly identified with 
the tangent bundle TOa equipped with the complex structure Ja- 

Let h be the Hermitian structure on TOa defined by : h(v,w) = FIa(v, Jaw) — 
iflA(v,w) for v,w £ TM. The symbol 

Thom(0 A , Ja) G T (0 A ,hom(p*(A^ e "T0 A ), /(A^TO A ))) 
at (m, v) £ TOa is equal to the Clifford map 

(3.15) Cl m (v) : A^ e "T m O A — A^T m O A , 

where C\ m (v).w = v A w — Ch(v)w for iu £ A'T^Oa- Here c^(u) : AjT m 0A — * 
A ,_1 T m OA denotes the contraction map relative to h. Since C\ m {v) 2 = — |v| 2 Id, the 
map Cl m (v) is invcrtible for all v ^ 0. Hence the characteristic set of Thom(0A, Ja) 
corresponds to the 0-section of TOa- 

It is a classical fact that the principal symbol of the Dolbeault-Dirac operator 
dc A + d*c A is equal tcQ 

(3.16) r A := Thom(0 A , Ja) ® C A , 

see [11]. Here also we have Char(TA) = — section of TOa- So ta is not an elliptic 
symbol since the coadjoint orbit Oa is non-compact. 

Following [33J [32), we deform ta in order to define a A-transversaly elliptic 
symbol on Oa ■ Consider the moment map $k ■ Oa —> 6* • With the help of the 
A-invariant scalar product on t* induced by the Killing form on q, we define the 
A-invariant function 

|| $ K || 2 :0a^K. 

Let TL be the Hamiltonian vector field for i= || || 2 , i.e. the contraction of the 
symplectic form by 7i is equal to the 1-form =£-d \\ || 2 . The vector field H 
has the following nice description. The scalar product on t* gives an identification 
E* ~ t, hence §k can be consider as a map from Oa to i. We have then 

(3.17) Hm = {V$ K (m))\ m , m£0 A , 

where V§ic{iri) is the vector field on Oa generated by &k (m) £ 6. 



Here we use an identification T*Oa — TOa given by an invariant Riemannian metric. 
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Definition 3.2. Let ta be the symbol on Oa defined in i3.16\) . The symbol ta 
pushed by the vector field TL is the symbol oa defined by the relation 

er A (™, v) := T A (m, v - TL m ) 

for any (m,v) G TOa- 

The characteristic set of a\ corresponds to {(m, v) G TO a, v = TC m }, the graph 
of the vector field TL. Since TL belongs to the set of tangent vectors to the if -orbits, 
we have 

Char (oa) H TkOa = {(m, 0) G TO A , H m = 0} 

= {me O a , d \\ <P K \\ 2 m = 0} . 

Therefore the symbol a a is if-transversaly elliptic if and only if the set Cr(|| || 2 ) 
of critical points of the function || <&k | 2 is compact. 
We have the following result. 

Lemma 3.3 ([TU1I32). The set Cr(|| <S> K || 2 ) C O a is equal to the orbit K ■ A. 

Corollary 3.4. The symbol a a is K -transversaly elliptic. 

3.3. Computation of Indcx^ (oa): the direct approach. The equivariant index 
of the symbol a a can be defined by different manners. 

On one hand, since Oa can be imbedded if-equivariantly in a compact manifold, 
one can consider IndexQ A (oA) G R^ C °°(K). 

On the other hand, for any if-invariant relatively compact open neighborhood 
U C Oa of Cr(|| $k \\ 2 ), the restriction of a a to U defines a class cta|w G Kjf(TjfW). 
Since the index map is well defined on U, we can take its index Index w (oa|w)- A di- 
rect application of the excision property shows that Index OA (oA) = Indcx^ {o~a\u)- 
In order to simplify our notation, the index of a a is denoted 

Indcx K (oa) G Rtc°°(K). 
The aim of this section is the following 
Proposition 3.5. Let A G Choi- We have 

Index A '(o A ) = S(p+) ® Va in Rt c °°{K). 

The rest of this section is devoted to the computation of Index if (oA). A similar 
computation is done in Section 5.2 of |34j in the context of a "Spin" quantization. 
Let 

(3.18) T : Oa — ► 0' A := K ■ A X p 

be the if -equivariant diffeomorphism defined by : T(g - A) = (fc • A, AT) where 
g = e x k, with k G K and X G p, is the Cartan decomposition of g G G. 

The data (^a, Ja, £a, TL, o~a), transported to the manifold O'^ through T, is 
denoted (Q' A , J' A , C' A ,TL' ,cr' A ). It is easy to check that the line bundle C' A is the 
pull-back of the line bundle K Xk a Ca — > K ■ A to 0' A . 

We consider on 0' A the following AT-equivariant data: 

(1) The complex structure J A which is the product Jk a x ~ad(z ). Here Jk a 
is the restriction of J a to the Kahlcr submanifold K ■ A C G ■ A, and ad(z Q ) 
is the complex structure on p defined in the introduction. 

(2) The vector field TL" defined by: H'£ x = -(0, [£, AT]) for ^ G K ■ A and 

x ep. 
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Definition 3.6. We consider on 0' A the symbols: 

• t'I :=Thom(C A! J^)®£^ 

• a" A which is the symbol t a pushed by the vector field TL" (see Def. \3.2\) . 

Proposition 3.7. • The symbol a" A is a K -transversaly elliptic symbol on 0' A . 

• IfU is a sufficiently small A -invariant neighborhood of A ■ Ax {0} in 0' A , the 
restrictions o' A \u and o" A \u define the same class in Kk(T^-W). 

Proof. The first point is due to the fact that the vector field TL" is tangent to the 
if -orbits in 0' A . Hence 

Char«) n T k O' a ~ {(£, X) e 0' A \ = 0} 

= A ■ A x {0}. 

Here we have used that [£, X] = for £ £ A' ■ A and A 6 p if and only if X = 0. 

We will prove the second point by using some homotopy arguments. First we 
consider the family of vector fields TLt ■= (1 — t)Tt' + tTL" , t G [0, 1]. Let a t be the 
symbol r' A pushed by TLt . On checks easily that there exists c > such that 

(3.19) nl x =H'l x + o(\\X\\ 2 ) and || f> c \\ X f 

holds on 0' A . With the help of (|3.19[) it is now easy to prove that there exists a 
A-invariant neighborhood V of K ■ A x {0} in 0' A such that 

Char(cr t |v) n T K V = A • A x {0}. 

for any t G [0, 1]. Hence a A \u = aolu defines the same class than u\\u in Kx (T#£/) 
for any A-invariant neighborhood U of K ■ A x {0} that is contained in V. 

In order to compare the symbols o' A \u and <Ji\ui we use a deformation argument 
similar to the one that we use in the proof of Lemma 2.2 in |33j . 

Note first that the complex structures J' A and J A are equal on A • A x {0} C 0' A . 
We consider the family of equivariant bundle maps A u € T(0' A , End(TO^)), u £ 
[0, 1], defined by 

A u := Id -uJ' A Jl 

Since A u = (1 + u)Id on A • A x {0}, there exists a A-invariant neighborhood W of 
A ■ A x {0} (contained in V), such that A u is invertible over U for any u G [0, 1]. 

Thus A Ul u G [0, 1] defines over U a family of bundle isomorphisms : A$ = Id 
and the map A\ is a bundle complex isomorphism 

M ■ (TW,4) — > (TW,J A ). 

We extend Ax to a complex isomorphism A§ : Aj'^TU — > hj^TU. Then in- 
duces an isomorphism between the symbols Thom(W, J A ') and Ai*(Thom(£/, J' A )) : 
(x,v) i — * Thom(W, J' A ){x, Ai{x)v). In the same way induces an isomorphism 
between the symbols a A |^ and j4i*(cti|^) : (a;, u) i— > r A (a;, Ai(a;)(w — TL'ff). One 
checks easily that A u *{o-[\u), u G [0, 1] is an homotopy of transversaly elliptic sym- 
bols. 

Finally we have proved that & A \u, an( l a 'k\u define the same class in 

K K (T K U). a 

Here also, the equivariant index of the transversaly elliptic symbol o~ A can be 
defined either as the Index^/ (<r A ) taken on 0' A , or as the index Index^(er A |^) taken 
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on any ^-invariant relatively compact open neighborhood U C 0' A of K ■ A x {0}. 
We denote simply 

Indcx^K) G Rtc°°(K)- 
the equivariant index of cr A . The second point of Proposition 13.71 shows that 
Index^(cr^|jY) = Index^"(cr A |^). Hence we know that 

Indcx^CTA) = Indcx^ (cr'Jl) ■ 
In order to compute Index^ (<7y(), we use the induction morphism 

defined by Atiyah in pQ (see also [33] [Section 3]). The map enjoys two properties: 
first, j» is an isomorphism and the AT-index of a € Kjf(Tx(OJ\)) can be computed 
via the i^A-index of (cr). 

Let a : p*(E + ) — > p*(E~) be a AT-transversaly elliptic symbol on A , where 
p : TO A — > C A is the projection, and E + 7 E~ are equivariant vector bundles over 
C A . So for any (£, A") G K ■ A x p, we have a collection of linear maps <r(£, AT; u, Y) : 
^ x) — * ®(i A') depending on the tangent vectors (y, Y) G T^(K • A) x p. The 
ifA-equivariant symbol j~ (cr) is defined by 

(3.20) V)(*> Y) = a(A, X; 0, Y) : E+ A x) — > £ (A >Jf) for any (X, Y) € Tp. 

In the case of the symbol cr A , the super vector bundle E + © E 1- over O a is 
A*«T0 A ® £ A . For any X G p, the super vector space x ^ © x ^ is equal to 

Let Thom(p~) be the Thorn symbol of the complex vector space p~ ~ (p, — ad(z )). 
Let A be the vector field on p~ which is generated by A G t* ~ t. Let 

Thom A (p~) 

be the symbol Thom(p~) pushed by the vector field A (see Definition l3.6j) . Since the 
vector field A vanishes only at € p~, the symbol Thom A (p~) is A'A-transversaly 
elliptic. We have 

(3.21) (j*rV A ) = Thom A (p-) (g> A£f/« A 8> C A . 

In (|3.2ip . our notation uses the structure of R(K\)- module for Kk- a (T#- a P), 
hence we can multiply Thom A (p") by Aje/6 A ® C A . 

Let C~°°{K\) Ka , C _oc (A') i< " be respectively the vector spaces of generalized 
functions on K\ and K which are invariant relative to the conjugation action. Let 

(3.22) Indf A : C-°°(K a ) Ka — ► C^if)* . 

be the induction map that is defined as follows : for (j> G C~°°(K\) Ka , we have 

/ Ind£ A (0)(fc)/(fc)dfc= V f ( { ^ d , k l, f Hk')f\KAk')dk', 
Jk vol(AT A , rffc') 7a a 

for every / G C°°{K) K . 
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Theorem 4.1 of Atiyah in [Tj tells us that 



(3.23) K KA (T KA p) K k (T k O' a ) 



Index KA 



Index* 



C-°°(K a ) Ka ^C-°°(A) K . 

is a commutative diagranfl. In other words, Index* (<r) = Ind^ A (Indcx KA (j~ 1 (cr))). 
With (j3~2T|) we get 

Index* «) = Indf A (lndcx* A (Thom A (p-)) ® Ajt/*A ® C A 

We know from [33][Section 5.1] that the AA-index of Thom A (p~) is equal to the 
symmetric algebra 5(p + ) viewed as a AA-module. Since 5*(p + ) is a A-module, we 
have 

Index* K) = Indf A (5(p+)|x A ®AjB/*A®C A ) 
= 5(p+) 8 Ind| A (Aj«/e A ® C A ) 
= 5(p+)®Vf. 

The proof of Proposition 13 . 51 is then completed. See the Appendix in [33] for the 



relation Ind£ A (A' c t/t A <8> C A ) = 

3.4. Computation of Index* (<r A ): the shifting trick. This section is devoted 
to the proof of the following 

Proposition 3.8. Let 0a be the coadjoint orbit passing through A € Choi- 

For any ji E K, the multiplicity of in Index (<t a ) is equal to Q((0\)^). In 
other words we have 

Index*(cr A ) = Q k °°(O k ). 

The proof, which follows the same line of Section 4.1 in [34], starts with the clas- 
sical "shifting trick". For any V £ R~°°(K), we denote [V] K E Z the multiplicity 
of the trivial representation in V. 

By definition the multiplicity m\(n) of V? in Index* (<r A ) is equal to 
[Index* (ct a ) ® (V*)*]* where (V^*)* is the (complex) dual of V* . The Borel- 

at the representation V K 

Ricmann-Roch character 

RR K {K-ii,C M ), 

where Cr^i ~ K Xf p C M is the prequantum line bundle over the coadjoint orbit 
A ■ ii. Note that A ■ \x is equipped with the Kahler structure (fijf.^, Jk-p) where 
SIk-h is the Kirillov-Kostant-Souriau symplectic form, and Jk ^ is the A-invariant 
compatible (integrable) complex structure. 



Weil Theorem tells us that the representation is equal to the A-cquivariant 



Hence the dual (V* )* is equal to RR (K ■ fj,, C[_^j ), where A ■ \i is the coadjoint 
orbit A ■ \i equipped with the opposite Kahler structure (— Or-.^, — Jk-h)- Let 
Thom(A' • n) be the equivariant Thorn symbol on (A • [/,, —Jr-v)- Then (V?)* is 
equal to Indexj£ (Thom(A • /i) ® C[_ M j). 



2 Here wc look at R- C °°(K A ) and R- C °°(K) as subspaccs of C-°°(A'a) a " a and C^ 00 ^)* by 
using the trace map (see 13. 131 . 
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Let Thom(0 A ) be the Thorn symbol on (0a, Ja)- Like in section [321 let TL be 
the Hamiltonian vector field of =± || $ K II 2 : O a -> R. We denote by Thom w (C A ) 
the symbol Thom(0 A ) pushed by the vector field TL : 

Thom w (C A )(TO,i') := Thom(0 A )(m, v - TL m ), (m,v) £ TO A - 

Since Indcx K (cr A ) is equal to IndexQ A ^Thorn^C^) £ A J , the multiplicative 

property of the index pQ [Theorem 3.5] gives 
(3.24) 

m A (/i) = [indcx^^ ((Thorn* (0 A ) ® £ A ) (Thom(A^I) <g> C[_ m] ] 

See [J [33] , for the definition of the exterior product 

: K K {T k Oa) x K K (T(K • fx)) K K (T*(0 A x A ■ y)). 



K 



It is easy to check that the product Thom(0 A ) Thom(A ■ fx) is equal to 
the Thorn symbol Thom(C A x K ■ fx) on the manifold (C A x K ■ fx, J A x —Jk-h)- 
Hence the product Thom w (C A ) Thom(A ■ fx) is equal to the Thorn symbol 
Thom(C A x K ■ fx) pushed by the vector field (TL, 0) : let us denote it 
Thom (w,0) (O A xTC^Jx). 

The tensor product 

C:=Ch® C[_ M ] 



is a prequantum line bundle over the symplectic manifold C A x A ■ fx. 
Finally (|3.24|) can be rewritten as 

(3.25) m A (/x) = Indcxg AXK . ;i (Thom (W ' 0) (e> A x A~^) C) 1 ' 



The moment map relative to the Hamiltonian A-action on C A x A ■ fi is 



<!>! : C A X K ■ fx — > {* 
(3.26) (m,0 •— ► $if(m)-C 



For any f 6 I, we consider the map 3> t : Oa X A' • fx — > 8*, $t(m, £) := $(m) — i £. 
Let Ti t be the Hamiltonian vector field of ^ || <I> t || 2 : C A x A ■ ll — > R. We denoted 
Thom Wt (C A x A' • /x) the symbol Thom(C A X K ■ fx) pushed by the vector field Tit- 

We have the fundamental 

Proposition 3.9. • There exists a compact subset K. of C A such that 

Cr(|| $ t || 2 ) cfCxK-fx 

for any t £ [0, 1] . 

• TTie symbols Thom (w ' o) (0 A x TC^~fx) and Thom Wt (C A x ~K^~f~x),t £ [0,1] are 
K -transversaly elliptic. 

• The symbols Thom (w - 0) (£> A x A ■ fx) and Thorn"* (£> A x K ■ fx),t £ [0, 1] define 
the same class in Kx(T^(O a x K ■ fx)). 

Proof. The proof of the first point is given in [M] [Section 5.3.] when A is regular. 
In the Appendix, we propose another (simpler) proof that we learn from Michclc 
Vergne. For the second point we check that 

Char (Thom (w - o) (0 A x A~^)) n T K (O a x A~^)) ~ Cr(|| $ K f ) X K ■ fx 
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and 



Char (Thom Wt (e> A xK^)|nT K (0 A xi(. fi)) ~ Cr(|| $ 



II 2 ) 



are compact subsets of the 0-section of T(Oa x K ■ li). 

Since Cr(|| $ t || 2 ) stay in the compact set K, x K ■ fi for any t G [0, 1], the family 
Thorn 7 ** (£> A x ~K~fl) is an homotopy of transversaly elliptic symbol : hence they 
define the same class in K^-(T/ < -(Oa x K ■ fj,)). 

The vector field Ho on Oa x K ■ li is equal to (H, V) where 

V(m,0eT c (ff./i) 

for any (m,£) G Oa x K ■ Li. We use the deformation (7i, sV),s G [0,1]: let 
Thom (w ' sV) (0 A x iT~pO be the Thorn symbol pushed by the vector field (H, sV). 
It is easy to check that there exists a compact subset of K! C T(Oa X K ■ fi) such 
that 

Char (Thom (w ' sy) (C»A x iT 7 ^)) D T K (O a x iT 7 ^)) C AC' 

for any s G [0,1]. The family Thom (w ' sy) (0 A x ~K~l7),s G [0,1] is then an 
homotopy of transversaly elliptic symbols : hence Thom (W ' 0) (e> A x K ■ ll) and 
Thom Wo (0 A x ~K~l7) define the same class in K k (T k (O a x K ■ li)). □ 

Following the former proposition and (|3.25[) . we have 

(3.27) m A ( M ) = [index^x^ (Thorn 7 * 1 (C? A x if 7 ^) ® £)]* • 

We are now in the following setting : 

• X := Oa x K ■ ll is a Hamiltonian if- manifold with a proper moment map 

$i : x -> r, 

• £ is a prequantum line bundle over X , 

• the Hamiltonian vector field Hi of the function =^ || $i || 2 vanishes on a 
compact subset. 

Hence the "pushed" Thorn symbol Thorn 7 * 1 (A") is if -transversaly elliptic on X. 
In this context we can consider the equivariant index Index (Thorn 1 (X) ® C) , 
and we have the following theorem 

Theorem 3.10 ( 33, 351). The multiplicity of the trivial representation in 
Index^ (Thorn 7 * 1 (X) (8> C) is equal to Q(Xq), where Xq is the (compact) symplectic 
reduction of X at 0. 



If we apply Theorem 13. 101 to X = Oa x K ■ li we have Xo ~ (Oa)//, and then we 
can conclude that 

m A (/x) = Q((0 A )/ 1 ). 
The proof of Proposition 13 . 81 is then completed. 



4. PROPERNESS AND ADMISSIBILITY 

In this section, we consider an element A G Choi, and a closed connected subgroup 
H of K. We consider the representation Vjf <g> S^p" 1 ") of K: note that it is an ad- 
missible if-representation since the circle group Z(K ) acts on p + by multiplication. 
We are interested in the condition 

CI The representation <E> S(p + )\n is an admissible H — representation. 
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Let $/f : Oa — > f)* be the moment map relative to the Hamiltonian action of i/ 
on the coadjoint orbit Oa := G • A: the map $// is simply the composition of the 
moment map <&k '■ Oa — * 6* with the canonical projection it : I* — > f)*. 

Let us consider the condition 

C2 The map : A — > fj* is proper. 

The aim of this section is to prove that CI •<=>■ C2. During the proof, we will 
obtain other equivalent conditions. 
We start with the 

Lemma 4.1. We have C2 =>• CI. 

Proof. We have proved in Section that <g> S(p+) is equal to Q^°°(£>a)- Then, 
Property P2 of Theorem 12.71 tells us that the properness of implies the in- 
admissibility of Vjf <8> S(p + )\h ■ Since this fact is easy to prove, let's recall it. 

For \i 6 K and v 6 H we denote iV/f = dim(honiff (V^ , V^" | ) ) the multiplicity 
of Vj 1 in the restriction V*\h. Since 1// S(p+) = E^k 2( m m) wc know 
that the multiplicity (possibly infinite) of Vj 1 in ® S{\> + ) is 

(4.28) £j\tfe(M„). 

Let us see that the former sum is always finite when C2 holds. Since V? is 
equal to the /T-quantization of the coadjoint orbit K • fj,, the restriction V^\h 
is equal to the quantization of K ■ fj,, viewed as a Hamiltonian //"-manifold: the 
corresponding moment map K-fi — > h* is the restriction of the projection 7r to K-\i. 
The "quantization commutes with reduction" theorem tells us that the multiplicity 
Njf is equal to the quantization of the symplectic reduction of the Hamiltonian 
//-manifold K-/J, at v. In particular Njf ^ implies that v £ tt(K ■ fi). Finally 

N?Q(M„) + =>■ n e K- tTV) and ^(/i) ^ 0. 
These two conditions imply that we can restrict the sum of (|4.28|) to 

which is finite since &h is proper. □ 

The rest of this section is dedicated to the proof of CI C2. Since V.? is 
finite dimensional, one notices that CI is equivalent to : 

Cl' The representation 5(p + )|# is an admissible H — representation. 

4.1. Formal quantization of p. Let us denoted p~, the real vector space p 
equipped with the complex structure — ad(z ) (see the introduction). Let ft p be 
the (constant) symplectic structure on p defined by 

(4.29) Q p (X, Y) = B s (X, ad(z )F) 

where B g is the Killing form on g. 

Let h be the Hermitian structure on p~ defined by h(X,Y) = B(X,Y) — 
iQp(X,Y). Let U := U(p~) be the unitary group with Lie algebra u. We use 
the isomorphism e : u — > u* defined by (e(A),B) = -Trc(AB) <E R. For X, Y e p, 
let X ® Y* : p" -> p" be the C-lincar map Z h-> h(Z, X)F. 
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The action of U on (p, P ) is Hamiltonian with moment map $u : p — > u* defined 

by 

(*u{X) t A) = Slp(A(X),X), lep.ieu. 
Via g, the moment map $u is defined by 

(4.30) <S>u(X) = -X <S>X*, Xep. 

The Hamiltonian space (p,Op,<I>u) is prcquantized by the trivial line bundle, 

equipped with the Hermitian structure (s,s')\x = e "2 " ss' and the Hermitian 
connexion V = d — i6 where is the 1-form on p defined by = £lp(X, dX). 

One sees that $u is a proper map. Hence we can consider the formal quantization 
<2u°°(p, O p ) e i?~°°(U) of the U-action on the symplectic manifold (p, O p ). We are 
also interested in <2u°°(p, fcO p ) £ R~°°(\J), for any integer k > 1. 

Lemma 4.2 ([35)). TTie symmetric space S(p + ) is an admissible U -representation. 
The following equality 

(4.31) Q u 00 (p,m p ) = 5(p+) 
holds in R~°°(U), {or any k > 1. 

Proof. In |35j . we consider the case fc = 1. The other cases follow since the sym- 
plectic vector space (p, fcO p ) is equivariantly symplectomorphic to (p, Op). □ 

4.2. Formal quantization of p relative to the if -action. The adjoint action 
of K on p defines a morphism 92 : K — ► U. Let us denoted by tp : i — ► u the 
corresponding morphism of Lie algebra, and by <y9* : u* — > 6* the dual linear map. 
The moment map <S>jf : p — ^ 6* is equal to the composition of $u : p — > u* with ip* . 
Via the identification 6* ~ t givenj by the Killing form B g , the moment map $^ 
is defined by 

(4.32) & K (X) = -[X,[z ,X\] el, Xep. 

We note that « ) = || [z OJ X] || 2 > if X 0. Hence the moment map 

$x : p — t* is proper. We use property[P2] of Theorem 12 .71 fsee also Appendix C) 
to get from Lemma l4~2l the 

Corollary 4.3. The symmetric space S(p + ) is an admissible K -representation. 
The following equality 

(4.33) Q7/°(p,M7 p ) = S(p+) 
holds in R~°°(K), for any k > 1. 

We look now at the Hamiltonian action of a closed connected subgroup H C K on 
(p, Op). The moment map : p — > f)* is the composition of the map <I>x : p — > I* 
with the canonical projection 7r : — > t)* . In this setting, we know from property 
[P2] that the propcrncss of implies that S(p + )\n is an admissible representation 
of H. In [35] [Section 5], we have proved the converse. Let (p) be the Kirwan 
polyhedral convex set associated to the Hamiltonian action of K on (p,O p ). Let 
f) 1 - := ker(7r) C I*. We have the 

Lemma 4.4 (|35|). The following conditions are equivalent : 



3 The map £ G t* t—¥ £ £ 6 is defined by the relation {£, X) 
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(1) A K (p)nK-t, ± = {0}, 

(2) the map &h '■ p — > h* * s proper, 

(3) The subalgebra S(p + ) H formed by the H-invariant elements is reduced to 
the constants. 

(4) CI' : S(p + )\h is an admissible representation of H . 

Proof. Since the map &h : p — > f)* is quadratic, the map is proper if and only if 
$^(0) = {0}. Now it is easy to check that $ K ($^(0)) = K ■ A K (p) n f)- 1 . Hence 
(1)^<I>^(0) = {0}^(2). 

The equivalence of the last three points uses property [P2] and some basic results 
of Geometric Invariant Theory (see Lemma 5.2 in [35]). □ 

4.3. Proof of CI C2. Let Ak(O a ) be the Kirwan polyhedral convex set 
associated to the Hamiltonian action of K on (0a, ^a)- 

To any non-empty subset C of a real vector space E, we associate its asymptotic 
cone As(C) C E formed by all the limits y = Hindoo t k y k where (tf~) is a sequence 
of non-negative reals converging to and yk £ C. Recall the following basic facts: 

(1) As(C) is a closed cone, 

(2) As(C) = {0} if C is bounded, 

(3) As(C) = C if C is a closed cone, 

(4) If a compact Lie group K acts linearly on E, then As(K ■ C) = K ■ As(C). 
Proposition 4.5. Let A £ Choi- We have 

A k {O k ) = A K (K • A x p) and As (A k (O a )) = A K (p). 

Proof. For any integer k > 1, the coadjoint orbit O a , equipped with the symplcctic 
form kfl A , is symplectomorphic to (O kA ,fl kA ). Theorem 13.11 shows then that 

Q K °°(o A ,kn A ) = v k K A ®s(p + ). 

Consider now the product O a := K-Axp equipped with the symplcctic structure 
Cl' A := £Ik- A x Qp'- here £Ika is the Kirillov-Kostant-Souriau symplcctic form 
and ftp is defined in (|4.29| . For any integer k > 1, the symplectic manifold is 
(0' A , kfl' A ) is pre-quantized by (£ A )® fc , where C' A is the pull-back of the line bundle 
K x Ka C a -> K ■ A to Q' A . 

Since (p, ri p ) has a proper K- moment map, we can use property [P3] of Theorem 
[2771 We have 

Q K °°{0' A ,m' A ) = Qk(K ■ A, kilK-fi) ® Q K °°{p, kilp) 
= V k K A ®S(p + ). 

We are now in the setting of Lemma |2~51 : (O a , Sl A ) and (O a , fl' A ) are two pre- 
quantized proper Hamiltonian A'-manifold such that Q K °°(0 A7 kQ A ) = 
Qr K °° (0' A , kCl' A ) for any integer k > 1. This implies that A k (O a ) = A K {0' A ). 
Hence the first point is proved. 

For the other point, we first observe that A + Ak (p) C Ak(0' a ), so 

A K (p) = As (A + A K (p)) c As (A K {0' A )) . 

Let y £ As (A K (0' A )). We have y = lim^oo t k y k with y k = y' k + y' k \ where 
y' k £ K ■ A, y k £ (f>^(p), j/^, + £ t*j_ and t k is a sequence of positive number 
converging to 0. Since y' k is bounded, we have 

y= lim t fc i4' £ $^(p)nt;. 
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So we have proved that y G A^ (p). With the first point we can conclude that 
A K (p) = As (A K (0' A )) = As (Ajc(Oa)) • 

□ 

Remark 4.6. When A G Choi is K -invariant, the Kahler manifold Oa is exactly 
the Hermitian symmetric space G/K. In this situation, McDuff fZT\ has shown that 
G/K is symplcctomorphic to the symplectic vector space (p,fi p ). 

In the light of Proposition [775[ we conjecture that for any A G Choi, the coadjoint 
orbit Oa is K -equivariantly symplcctomorphic to the product K ■ A x p equipped 
with the symplectic structure fix- A x ^p- 

We need the following basic 

Lemma 4.7. Let (M, SI) be a Hamiltonian K -manifold with a proper moment map 
: M — > 6* . Let H C K be a closed connected subgroup. Let <5>h '■ M — > t* 6e i/ie 
moment map relative to the action of H on M. Suppose that we have 

As (Ajf(M)) flA'-l) 1 = {0}. 

TTien i/iere exists c > smc/i £/ia£ ||$#(m)|| > c|[$jf (m)|| ZioZcfc outside a compact 
subset of M . In particular <$>h is a proper map. 

Proof. Suppose that there exists a sequence m, G A/ such that 

lim W^KimM = oo and lim ||^4I = 0. 

We put $k (nii) = ki ■ yi with ki £ K and y« G A^ (M). Wc have then 

lim n [ki - -rf^-- ) = 

where 7r : £* — > f)* is the projection. Here we can assume that the sequence ki 
converge to k G K, and that the sequence ^-jj- converge to y G As(A#-(M)), 
||y|| = 1. We get then that 7r(fc • y) = 0. In other words, y is a non-zero element in 
As (Ajr(M)) D • ker(7r). □ 

We can now finish the proof of CI =>• C2. We have already check in Lemma 
[jl] that 

CI <^> CI' ^ A K {p) n K ■ t) 1 - = {0}. 

Wc have proved in Proposition ^. 51 that A^ (p) = As (Ajj- (Oa)), so condition CI 
is equivalent to 

(4.34) As(A K (O A ))nK-t, ± = {0}. 

Finally, we know after Lemma 14.71 that ()4.34j) implies the properness of the 
moment map : Oa — > f)*. 
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5. Description of Ak(P) 

The purpose of this section is the description of the Kirwan polyhedral cone 
Aa'(p) which is attached to the Hamiltonian action of K on (p, fi p ). 

For any root a G SK = SH(gc, tc) the corresponding root space g a C flc is defined 
as {X e 0c | [H, X] = i{a, H)X, V H e t}. 

For the rest of this section, we work with the system of positive roots JHjtj = 91^" U 
$K+' Z ° defined in the introduction. For any positive non-compact root /3 G 9^t' 2 °, 
there are if ,3 G t, Ep G 0/3, £L/3 G such that 

(5.35) ^ = E_p 

9 

B s (Ep,E-p) - 



II/3II 2 

Here J hi 1 is the conjugation on gc relative to the real form g, and the norm 
|| — || 2 on t* is induced by the Killing form B g . 

Note that conditions (|5.35| implies that [iHp,Ep] = 2Ep, [iHp,E_p] = —2E_p 
and 

(5.36) Hp * -2^ 

through the isomorphism t ~ t*. In particular iHp, Ep and E_p span a subalgebra 
of gc isomorphic to s[(2, C). 

For /3 G ^+' 2 °, let Xp = \(Ep + E_p) and Yp = ±(Ep - E_p). Thus the set 
{Xp, Yp}p e tn+-*o is a real basis of p. Since (fl,z ) = 1 for any (3 G 5H^' Z °, we have 
&&{z )Xp = —Yp and ad(z )Yg = Xp. 

We will now describe the restricted root system of G/K . Two roots a, /? £ 9t are 
strongly orthogonal, written aJJ3, if neither of a ± /? is a root. One can easily check 
that strong orthogonality implies orthogonality with respect to the scalar product 
on t*. 

Consider the "cascade construction" 

\P = {71,..., 7 r }, maximal set constructed by : 
7! is the maximal root in 5H^' Zo 

Ji+i is the maximal root in {(3 G 9^j/ z ° | /31/yk for fc = 1, . . . , i}. 

For the roots 7^, we denote simply Xk 7 Yk,Hk the elements X lk ,Y lk , H lk . We 
have the classical result (see [T2][Prop. 7.4]) 



Lemma 5.1. T/ie subspace 



is maximal abelian in p. 



*:=i 



Since p = if • a, it is sufficient to understand the image of a by <&k to compute 
Ak(p) ■ in fact this Kirwan cone will be computed by describing the image by <f>x 
of a closed cone a+ C a, which is a fundamental domain for the if-action on p. 

For A G o*, we write 

A := {X G q I [H, X] = (A, H)X for all H G a}. 
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If g x 7^ and A ^ 0, we call A a restricted root of g. The set of restrited roots is 
denoted S. Let Wy, be the group generated by the orthogonal symmetries along 
the hypcrplane kcr(A), A G S. A proof of the following classic result can be found 
in [20] [Sec. VL5]. 

Proposition 5.2. • £ is an abstract root system on a*. 

• The group W-y is finite and is canonically identify with the quotient Nk (a)/ Zk(&) , 
where Nk(o) is the normalizer subgroup of a in K and Zx(a) is the centralizer sub- 
group of a in K . 

With the help of a system of positive roots S + , we define the closed chamber 
o+ := {H G a | (X,H) > for all A e £+} . 



Proposition 1 5 . 21 tell us then that any if -orbit in p intersects a+ in a unique point. 
We have the fundamental 

Proposition 5.3. For a particular system of positive roots S + ; we have 

r 

fc=i 

Proof. The proof is done in Appendix B. □ 

An element X of the chamber Y?k=i R^°(Ai + • • ■ + X k ) is of the form X = 
J2k=i tkXk with t\ > ■ ■ ■ > t r > 0. Then $>x(X), view as an element of 6, is equal 
to 

*k(X) = -{X,[z ,X}} 



k,l 



2 ^ 

fe=i 



(t k ) 2 H k . 



Here we have used the fact [X k , Y{\ = for k ^ I since [g 7fe , fl± 7; ] = 0. When k = I, 
one sees that [X k ,Y k ] = | [E lk , £A_ 7 J = -\H k . 

Since the vector — \H k S t corresponds to jp^jp through the identification t ~ t*. 
We conclude that 

= X>> 2 ^ e '* 

for X = J2 r k=1 t k X k . 

Let tl C t* be the Weyl chamber defined the by the system of positive compact 
roots Let Choi C t+ be the Weyl chamber defined by the system of positive 
roots SHjhoj. The following proposition will be proved in Appendix B. 

Proposition 5.4. All the roots 7 fc have the same lenghts, and we have 

r 

Cho! n Vect( 7l , . . . , 7r ) = + ■ ■ ■ + ik)- 

In particular, the weight 71 + ■ • • + 7& is dominant for any k = 1, . . . , r. 
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We know then that <&k(X) — || 7 ^ J2k=i(tk) 2 7k belongs to the Weyl chamber t* 
if X = Y^k=i tkXk belongs to the chamber a+. Hence, the moment map §k '■ p — > 
t* defines a one to one map between a + and the the cone J2k=i R-°(7i + ■ ■ • + 7fc) C 
t?_. Using now the fact that a+ and t*j_ are respectively fundamental domains for 
the if -action on p and {*, we get the following- 
Proposition 5.5. • The Kirwan polyhedral cone Ak (jp) is equal to 

r 

(5.37) £ R >o (7l + ... +7fc) . 

fc=i 

• The K -Hamiltonian space (p, J7 p ) is without multiplicities : for any ( G t*, the 
fiber ^x X (K ■ £) C p is a K -orbit. 

We can summarize the results of Sections 0] and [5] in the following 

Theorem 5.6. Let ® 5'(p + ) be the admissible K -representation attached to 
A e Choi- ^ef H be a closed connected Lie subgroup of K. Let <&h : Oa — > fj* be the 
moment map relative to the action of H on the coadjoint orbit 0\. The following 
statement are equivalent: 

(1) The map &h : 0\ f)* is proper. 

(2) The H -multiplicities in (g> ^(p -1- ) are finite. 

(3) The subalgebra S(p + ) H formed by the H -invariant elements is reduced to 
the constants. 

(4) We have 

(5.38) r£ M^°( 7l + • • • + 7fc )) fl K ■ ^ = 

Remark 5.7. Note that the condition i5. 38\) holds trivially when H = K since 
then K ■ f) = {0}. When H is equal to the center Z(K) C K, the set K ■ f) = 
Lie(Z(if)) intersects Y^k=i ®~°(7i + • • • + 7fc) on/y at since (7^, z Q ) = 1 /or a/Z 
= 1, . . . ,r. 



We finish this section by considering the example of SU(p, q), with 1 < p < q. A 
maximal compact subgroup of SU(p, q) is K = S(U(p) x U(g)). The maximal torus 
T C K is composed by the diagonal matrices. The dual of its Lie algebra is 

t* :={(x 1 ,...,x p+q )eW>+« I 5> 3 -=0}. 

i 

The vector space p + is the complex vector space M Pi9 (C) of complex pxq matrices. 
The action of K = S(U(p) x U(g)) on p+ = M M (C) is defined by (o,/i) • M = 
gMh' 1 . 

The Weyl chamber relative to a system of positive compact roots £H+ is 
t* := {(xi, . . . , x p+q ) G t* I > ■ • • > x p and x p+ i > • ■ ■ > . 
The Weyl chamber relative to a system of positive roots 5Hhoi is 

Choi := {(xi, x p+q ) e t* I £1 > • • • > x p > x p+ i >■■■> x p+q } . 
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A family of strongly orthogonal roots is "J = {71, . . . , 7 P } wher^l 

lj = e J ~ e P+q-j+l- 

Hence the cone J2k=i R-°(7i H h Ik) is equal to 

V := . . . , x p , 0, ■ ■■, , -x p , . . . , -Xi) I xi > ■ ■ • > x p > o|. 

q—p times 

Let us consider the normal subgroups SU(p) and SU(g) of K. If H = SU(p), it 
is not hard to see that 

f) x nt* = |(x ! ^ L x,yi,...,y g ) I px + ]T]% = o|. 

times ^ 

so [j 1 fl P contains the non-zero element (1, . . . , 1, 0, . . . , , —1, . . . , — 1). Thus wc 

p times q—p times 

know from Theorem 15 . 61 that 

(1) the holomorphic discrete series representations of SU(p, q) does not have a 
admissible restriction to SU(p), 

(2) the algebra S(M pxq (C)) has an homogeneous SU(p)-invariant element with 
strictly positive degree. 

Consider now the case where H = SU(q') with p < q. We see that 

fj^Ht* = |(xi,...,x p ,j/, . 




and that f) fl T> = {0}. From Theorem 15.61 we have then that, if p < q, 

(1) the holomorphic discrete series representations of SU(p, q) have an admis- 
sible restriction to SU(g), 

(2) the algebra S(M pxq (C)) does not have an homogeneous SU (^-invariant 
element with strictly positive degree. 

6. Multiplicities of the discrete series 

Let G be a real, connected, semi-simple Lie group with finite center. Let K 
be a maximal compact subgroup of G, and T be a maximal torus in K. For the 
remainder of this section, we assume that T is a Cartan subgroup of G. The discrete 
series of G is then non-empty and is parametrized by a subset Gd in the dual t* of 
the Lie algebra of T Q3]. 

Let us fix some notation. Let fH c C 91 C A* be respectively the set of (real) roots 
for the action of T on 6 <g> C and g <g> C. We choose a system of positive roots 91+ 
for SK C , we denote by t+ the corresponding Weyl chamber, and we let p c be half the 
sum of the elements of £H+ . 

An element A € t* is called regular if (A, a) ^ for every a G 91, or equivalcntly, 
if the stabilizer subgroup of A in G is T. Given a system of positive roots £H + for 
$H, consider the subset A* + | X) Q £5R+ « of t*. It does not depend on the choice of 
9t + , and we denote it by A* [5]. 

The discrete series of G are parametrized by 

(6.39) G d := {A e t*, A regular } n A* n t* . 



^Here {e\, . . . , e p + q } is the canonical basis of R p + 9 . 
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An element A E Gd determines a choice £H + ' A of positive roots for the T-action 
on q <g> C : a E 9^+< A (a, A) > 0. We have 9t+- A = m+ U 91+ ,A and we define 

Note that the Blattner parameter 

A(A) := A-/9 c + p„(A) 

is a dominant weight for any A E Gd- We work in this section under Condition 
(HU), which states that (3 E 9\+- x <^=> (/3, A(A)) > 0. This implies in particular, 
that the dominant weight A(A) does not belong to the non-compact walls. 

Let us consider the coadjoint orbit Oa(a) := G-A(X). It is a G-Hamiltonian man- 
ifold which is prequantized by the line bundle £\(\) ■= G X;{ t()| Ca(a)- We equip 
Ca(A) with the G-invariant almost complex structure J\(\) which is characterized 
by the following fact. The bundle T 1,0 O\^\) — > Oa(A) is equal, above A(A) E Oa(a)> 
to the T-module 

<a,A(A))>0 </3,A(A)><0 
P(A)" 

Similarly we note p(A)+ := X)/3em+' A fl/3 C p <g) C. Note that the almost complex 
structure Ja(\) ls compatible with the symplectic structure on Oa(a)j but in genaral 
Ja(a) is n °t integrable. 

Let H\ be a discrete series representation attached to A £ Gj. Recall that the 
restriction 7~1\\k is an admissible representation. 
The main result of this section is 

Theorem 6.1. If X E Gd satisfy condition il.5]) we have 

(6.40) Hx\k = Q k °°(O aw ). 

Like we did before, if we use (|6.40| together with the property [P2], we get 
Theorem [Ql 

The proof of Theorem l6.1l is similar to the proof of Theorem [3TTJ We introduce, 
like in Section 13 - 21 a if-transversaly elliptic symbol 0a(a) on ^A(A) built from the 
data (£a(a)i Ja(\)) an d the moment map <&k '■ ^A(A) The same deformation 

argument as the one used in Section [3.41 shows that 

Index* (ca(a) ) = Qk°°(Oa(x))- 
Thus Theorem 16.11 follows from the following 

Proposition 6.2. If X E Gd satisfy condition U.5)) . we have 

Index* {o- m )=H x \K in R£°{K). 

6.1. Proof of Proposition l6.2l The proof is an adaptation to the proof of Propo- 
sition [331 Here we consider the the J^-invariant diffeomorphism 

(6.41) T : O aw — > 5 A(A) := K x A - A(A) p, 
defined by f(ke x • A(A)) := [k,X). 
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The data ( J\(\) i £a(A) , 0a(A))j transported to the manifold Oa(a) through T, 
is denoted ( J A (a) > £a(a)j <?a(a))- The line bundle £a(a) is the pull-back of the 
line bundle K Xr mx) C A (a) -> K ■ A(A) to C A (A)- 

The tangent bundle TOa(A) is -KT-equivariantly isomorphic to K Xjf A(A . (t A © Tp) 
where tA := [I, A(A)] is the if A (A)- mvar i an t complement of 6a(a)- 

Let J\ be the (linear) complex structure on the vector space p such that (p,J\) — 
p(A) + . Note that J\ is -Ka(A) -invariant since A satisfies condition (|1.5p . 

Let Jjc-A(\)\e be the (linear) -Ka(A) -invariant complex structure on the vector 
space ca defined by the Kahler structure Jk a(\) on the coadjoint orbit A • A(A). 

We consider on Oa(a) the following if-equivariant data: 

(1) The almost complex structure J' A such that 

J'A\{e,v) = Jk A(X) |e x ~J\ for every v e p. 

(2) The vector field H' defined by: H' [k v] = - (0, k ■ [A(A), «]) for [fc, w] e O a(a) . 

Definition 6.3. We consider on Ca(a) the symbols: 

• ? A(A) : = Tn0nl (°A(A)>^ A ) ® Av(A)> 

• <5a(A) w hich is the symbol V us hed by the vector field TL' (see Def. \3.2\) . 

Proposition 6.4. • The symbol &'a(X) * s a K -transversaly elliptic symbol on C A (A) • 
• IflA is a sufficiently small K -invariant neighborhood of K x^ A(A) {0} in Oa(\), 
the restrictions (ta(\)\u an d &'a{\)\u define the same class in K/ < -(T^-W). 

Proof. The proof works as the proof of Proposition [377] □ 

Proposition 16.41 shows that Index^(<7A(A)) = I n dcx K (c7A(A)) = Index^ (<t A( . A j ) . 
In order to compute Index^cr^^), we use the induction morphism 

i. : K Ka (T Ka p) — » Kjc(T x (O a(a) )) 

defined by Atiyah in [1] (see also [33] [Section 3]). Here i* differs from the induction 
morphism j„ used in Section 13. 3[ by the isomorphism 

K x (T k (5a(a))) - K X (T K (A'-A(A) x p)) 

induced by the if-diffeomorphism 0a(a) — K-A(X) x p, [k, X] \— > (k ■ A(A), k ■ X). 
Let Thom(p(A)~) be the i^A(A)-equivariant Thorn symbol of the complex vector 

space p(A)~ ~ (p, — J\). Let A(A) be the vector field on p which is generated by 
A(A) G 6* ~ B. Let 

Thom A(A) (p(A)-) 

be the symbol Thom(p(A)~) pushed by the vector field A(A) (sec Definition 13. Gl) . 
Since A(A) does not belongs to the non-compact walls (see condition (|1.5p ). the 

vector field A(A) vanishes only at G p: hence the symbol Thom A( - A ' ) (p(A) _ ) is 
-^A(A)"t ran sversaly elliptic. 
One checks easily that 

(6.42) (**rV A( A)) = Thom A ( A >(p(A)-) ® A£t/t A(A) ® C A(A) . 
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Let Indf : C~°°(K A (x)) Kaw — > C-°°(K) K be the induction map introduced 
in (|3.22[) . Equality (|6.42[) and the commutative diagram (|3 . 23[) give 

Index* (a A(A) ) = Indf a(a) (lndex* A W (Thom A < A >(p(A)-)) ® Aj«/«A(A) ® C A(A) ) 
= Ind£ (lndcx T (Thom A(A) (p(A)")) ® Ajl/t ® C A(A) 



In the last equality, we use two facts (see [53]) : 

• Since the symbol Thom A< - A -'(p(A) _ ) is T-transversally elliptic, the index 
Index^W (Thom A(A) (p(A)~)) is T-admissible, and its restriction to T is equal 

to Index (Thom A(A) (p(A)-)). 

• For any K\r x) -module E which is T-admissible we have 

Ind* A(A) [E ® A£«/« A (A)) = Ind^" (E\ T ® A#/t) . 

We know from [33] [Section 5.1] that the T-index of Thom A(A) (p(A)~) is equal to 
the symmetric algebra 5*(p(A) + ) viewed as a T-modulc. Here we use in a crucial 
way Condition (|1.5p : for every weight (3 relative to the T-action on the complex 
vector spaces p(A)~, wc have (/3, A(A)) < 0. The T-modulc S'(p(A) + ) is denoted 

n t 1 -^ 1 e r ^^- 

in [33]. So we have proved that 

(6.43) Indcx K ( ( r A(A) ) = Ind* I [ ]J (1 - t?)] ~* ® C A(A) ® Ajt/t 

Wc have proved in [34] [Section 5.1] that the Blattner formulas [15] which com- 
putes the A"-multiplicities of the discrete series representation Tlx are equivalent to 
the following relation 

(6.44) Hx\k = Hol£ I [ [] C 1 - ~* ® C A(A) in ^teW. 

where the "holomorphic" induction map Holy is equal to Indy (— ® AjS/t). 
We see that (|6.43j) and (|6.44| complete the proof of Proposition ^. 21 

6.2. Examples. 

6.2.1. The case of Sp(2,R). Wc examined this case in Example (|1 .3[i . Let 61,62 
be the Z-basis of of the lattice A*. The set of compact roots is 9t c = {±(#1 — #2)}, 
and the set of non-compact roots is 9l n = {±(#1 + #2), ±2#i, ±26 2 }- We choose 
61 — 62 as the positive compact root, hence t* = {6\ > 62}- 

The set of strongly elliptic elements in the Weyl chamber tl has four chambers 
(see Figure (16X11 ): d = {0 1 > 6 2 > 0}, C 2 = {6 1 > -6 2 > 0}, C 3 = {-0 2 > 61 > 
0}, and C 4 = {-0 2 > -fi > 0}. 

For A G t5_ which is regular, the term /0„(A) only depends of the chamber 
where A stands: let us denoted it p n (Ci). 

We check that — p c + p n (Ci) G C; for i = 2,3. Hence, for i = 2,3 and any 
Harish-Chandra parameter A G Cj, we have A(A) = A — p c + p n {Ci) G Ci. 
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FIGURE 2. Chambers for Sp(2,R) 



We know already that any regular weight of the holomorphic chamber C\ satisfies 
condition (| 1 .5() . It is also the case for the anti-holomorphic chamber C4. 

Finally we see that condition (| 1 . 5|) holds for any Harish-Chandra parameter of 
a discrete series of Sp(2,R). 

6.2.2. The case of Sp(4, R). Let l: ■ ■ ■ , 4 be the canonical basis of R 4 ~ t*. The 
compact positive roots are Q{ — 9j,l < i < j < 4, so that the corresponding Weyl 
chamber is t+ := {Ai > A2 > A3 > A4}, and p c = |(3, 1, — 1, — 3). The set of 
non-compact roots is {28i} U {8i + Oj,i < j}. 

We consider the chamber C := {Ai > A2 > — A4 > A3 > 0} of the Weyl chamber 
We have p n {C) = |(5, 5,3, -1) and then 

- Pc + Pn {C) = (1,2,2,1). 

We check that A = (5, 3, 1, —2) is a Harish-Chandra parameter belonging to C, but 
A(A) = (6, 5, 3, —1) does not belong to C. 

6.2.3. The case of SU(3,2). Let T be the torus of SU(3,2) formed by all the 
diagonal matrices. The dual of Lie algebra of T is t* = {(Ai,--- , A5) € R 5 | X^A, = 
0}. Let ei, • ■ ■ , es be the canonical basis of R 5 . The choice of positive compact roots 
fH^ is {ei — &2, ei — ei — es, — es} so that the Weyl chamber is 

t* := |Ai > A 2 > A 3 and A 4 > A 5 |. 

We have p c = (1,0,-1,^,—^). The non-compact roots are ±(e 2 ; — ej), i = 
1,2,3, i = 4,5. 

• Let A = (3, 1, —1, 0, —3) be in the chamber C\ := {Ai > A 2 > A 4 > A 3 > A5}. 
We have p n {Ci) = (1, 1, 0, —5, — |), and then 

A(A) = A - Pc + pnid) = (3, 2, 0, -1, -4) 

is a regular element which does not belong to C\. 

• Let us consider the chamber C2 := {Ai > A 4 > A 2 > A 5 > A 3 }. We see 
that p n {C2) = Pc, hence any Harish-Chandra parameter of the chamber C 2 satisfies 
condition ()1.5j) . 



32 



PAUL-EMILE PARADAN 



7. Appendices 

Let G be a connected real semi-simple Lie group with finite center. Let A be a 
maximal compact Lie subgroup of G. Let T be a maximal torus in K. Let t, I, g 
be the respective Lie algebras of T, A, G. We assume that t is a Cartan subalgebra 
of 0. 

In Appendix A, we use the identification X i— > B g (X, — ),g — > g* given by the 
Killing form. Hence the coadjoint orbits of G considered in the previous sections 
will be replaced by adjoint orbits. 

7.1. Appendix A. Let O = G ■ A be an adjoint orbit of G passing through A G t. 
Let K ■ [i be an adjoint orbit of A passing through /z G t. We consider the maps 

$ t : x A • /z — ► 6, i G [0, 1] 

defined by <j> t (m,£) = 7Tf(m) — t£. Here irt : g — ► I is the orthogonal projection. 
The maps $ f ,i G [0, 1] generates the vector fields Tit,t G [0, 1] on O x K ■ /z by 
Kt(n) = (V$ t (n))|„ for n G x A • p. 
The aim of this section is the following 

Proposition 7.1. There exists a compact subset K, of O such that 

{H t = 0} C K x K ■ pi 

for any t G [0, 1] . 

Proof. The proof is given in [34] [Section 5.3] in the case where A is a regular ele- 
ment of Q. Here we propose another proof, which is technically simpler, that was 
communicated to us by Michele Vcrgne. 
By definition, we have 

Wt(m,0 = -([Tt(m)-te,m] ) [7r l (m) ) f]) G T n OxT 5 (A- M ) 

Let us denote Ct the subset {Tit = 0}. We have 

C t = {(m,£) GO x K ■ (j, | [7r t (m) -1£,m] = and [7r t (ra),£] = 0} 

= A ■ {(m, /z) G O x K ■ [i | [7T{(m) — f/z, m] = and [nt(m), /z] = 0} . 

The condition [7rt(m),/z] = means that 7T{(m) belongs to the subalgebra that 
stabilizes /z G t. We have fi^ = A M • t, hence C t c A • P t x K ■ /z where 

P« = {m G O | 7r t (77z) G t and m G Wc ( m )_ ti u} ■ 

Here fl,r e (m)-tp is the subalgebra that stabilizes 7T{(m) — i/z. The proof will be 
settled if one proves that U tS [ .i]Pt is contained in a compact subset of O. 

The subalgebras Qx,X G t describe a finite subset that we enumerate Qi,i = 
1, ...,r. For each subalgebra jji, let Gi be the corresponding closed connected 
subgroup of G. Note that t is contained in each g i; and that the center of 
is contained in (the Cartan subalgebra) t. Note that the condition fl Wt ( m )_t^ = 0i 
implies that 7rj(m) — </z G z(fli)- It gives that X> t C yj r i=1 V\ with 

D| = {m G n & | 7r t (m) -t/z G z(fli)}. 

It is a classical result that the intersection n Qi is equal to a finite collection of 
adjoint Gi orbit: 

On 0l = (J G<-a. 

aGA; 
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Let TTi : q — > z($i) be the orthogonal projection. If TTt(m) — t\i £ z{Qi), we have 

7Tj(m) — tfX = TTi^lTt(m) — t/jlj 

= Wi(m) - i7Ti(/i). 
But the map 7Ti is constant on each connected component d ■ a. So finally, 
T>\ — {m £ G{ ■ a | 7T{(rn) — i/x = 7Tj(a) — t%i(fi).} . 

a£Ai 

and then 

c (J onv^Wi^t) 

a£Ai 

with 0i jQ .t = 7Tj(a) + — 7Ti(/i)). We get finally that 

(J i? t cOn^C) 

te[o,i] 

where 

C = {0 i>a ,t, t £ [0, 1], i = 1, . . . ,r, a e A,} 
is a compact subset of t. Since the map 7T{ is proper when restricted to O, the set 
fl 7r { " 1 (C) is compact. 

□ 

7.2. Appendix B. Here we suppose that G/K is an irreducible Hcrmitian sym- 
metric spaces, and we use the notations of Section [5l Our aim is the proof of 
Propositions 15.31 and 15.41 Our (classical) arguments uses the knowledge of the 
restricted root system E and the Cayley transform. 

We denote (— ,— )f the scalar product on t defined by : (X,Y) t := —B S (X,Y) 
for X, Y £ t. Let ( — , — ) t . be the scalar product on t* which make the map X i— > 
(X, —)t, from t to t*, unitary. 

Let a = Sj=i be the maximal abelian algebra of p attached to the maximal 
family $ = {71, . . . ,j r } of strongly orthogonal roots (see Section [5]). 

Let ti C t be the subspace orthogonal (for the duality) to the vector subspace 
spanned by 71, . . . , j r : ti is also the centralizcr of a in t. Let t2 C t be the orthogonal 
of ti (relatively to the scalar product on t). We check easily that 

t 2 =Vect(JT 1 ,... ) JT r ). 

We have then the orthogonal decomposition t* = t* © with = Vect(7i, . . . , j r ). 

Let *H = 9t({jc,tc) be the roots system associated to the Cartan subalgebra 
t = ti © t2- Let JHj^j = U 9\n' z ° be the system of positive roots consider in the 
introduction. Let Choi := {£, £ t* | (£,a)t* > 0, Va £ SH^oi} be the corresponding 
Weyl chamber. 

Let 7r' : t* — > be the canonical projection, and let us consider 
E':=7r'(<K)\{0} and (E') + := tt'^+J \ {0}. 
We see that C n Vect(7i, . . . , 7 r ) = V, with 
(7.45) V := {£ G t* 2 | (£, a) t . > 0, Va £ (E')+} . 

Now we use the description of E' given by Harish-Chandra and Moore. 
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Proposition 7.2 ( [T3l [3Q] ) . • All the 7 fc have the same lenght. 

• For any i < j, there is an a 6 !SH+, such that Tr'(a) = g(7i — 7j)- 

• TViey are iwo possibilities for E' := 7r'(£H) \ {0}: 

E' := { ± |( 7 i + 7j ), ±|( 7i - 7,), 1 < i < j < r} U { ± 7i , 1 < i < r}, 

or 

E':={±i(7 i +7 J -),±|(7i-7i), 1 < * < j < ^} U { ± \ lu ± 7i , 1 < i < r}. 

Since the j k belongs to (S') + , the last two point of Proposition shows that 

(7.46) (E') + = {|( 7 i + 7i). 5(7, ~ 7j)> 1 < * < 3 < r} U {71, • • ■ ,7r} US 

where S = or 3 = {± 7l) . . . , ± 7r }. 

Since the 7 & have the same lenght, it is now easy to see that the set D defined 
in (|7.45|) is equal to Y%=i K-°( 7 i + • • • + 7r ). Thus the second point of Proposition 
15.41 is proved: we have 

r 

(7.47) CnVcct( 7l ,..., 7r ) =^R> ( 7l + ...+7 fc ). 

k=l 

Remark 7.3. We know from \5.36\) that \\H k \\ = 2||7 fc ||~ 1 . Thus, all the H k have 
the same lenght. 

Now, we go into the proof of Proposition 15.31 we will compute a fundamental 
domain a + for the action oi K on p. 

In the complex semi-simple algebra u := t © ip, we consider the Cartan algebra 

fj := ti © ia. 

that we equip with the scalar product (X, Y)tj := —B g (X,Y), V X,Y 6 f). We 
take on f)* the scalar product such that the map Ij-i^Ih [X, — )(, is orthogonal. 

Let !EH(0c,f)c) C f)* be the set of roots relative to the adjoint action of fjc on 
0c- The projection it : rj* — > (ia)* sends 9t(0c,f)c) onto E U {0}, where E is the 
restricted root system, and £ 1— > £, a* ~ (ia)* is the one to one map defined by 
(liX) := (£,X). 

The Cayley transform 

c :=cxp ^-yad(^r fe )^J . 

is an automorphism of the complex Lie algebra 0c- One checks that c(Y) = Y for 
any Y G ti and that 

c(iX k ) = ^H k , Vfc = l,...,r. 

Hence the Cayley transform sends the subalgebra rj onto the subalgebra t. Moreover 
one checks easily that c : f) — > t is an orthogonal map, thus we know that all the 
X k have the same lenght. Let us denoted c* : t* — > f)* the dual orthogonal map. 

Since c is an automorphism of 0c, the image of the root system := 9l(0c,tc) 
by c* is equal to the root system £H(0c, f)c)- Since c is the identity map on ti, we 
have c*(E') = E. 

If we choose systems of positive roots such that 

5H(0 C ,f)c) + :=c*(K+ ol ) and E+ := c*((E') + ), 
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we got 

a+ := {X G a \(J3,X) > 0, V/3 G S + } 
= {X G a |(c*(a),zX) > 0, Va G 

= {X = J2a k X k \J2ak(a,H k )>0, Vae(E')+}. 

fc fc 

From the description (|7.46[) of we finally found that 

r 

fc=i 

7.3. Appendix C. Let ip : H ^ K be & morphism of compact connected Lie 
group. Let dp : fj — > t be the corresponding morphism of Lie algebras. Any 
Hamiltonian A- manifold (M, $x) can be seen as a Hamiltonian //-manifold, with 
moment map <$>h = rfy* <&K- 

The morphism ^ induces a map : R{K) — > R(H). When A G R~°°(K) is 
A-admissible (see Definition I2.6|) . one can define its "restriction" to H, that we 
denoty by ip*E (or simply A|#). 

The aim of this appendix is to check that the following version of [P2] holds. 

Proposition 7.4. Let M be a pre- quantized proper Hamiltonian A -manifold. If 
M is still proper as a Hamiltonian H -manifold. Then Q^-°°(M) is H -admissible 
and we have the following equality in A~°°(A) : 

Q k °°(M)\h = Qh°{M)- 

Proof. The proof is given in [35] when ip is the inclusion of a subgroup. Let us 
generalize this result to a general morphism tp : H — ► K. Let L := p>(H). We 
write ip — i o j where i : L K is the one to one map given by the inclusion, and 
j : H — ¥ L is the onto morphism induced by p. 

We consider the one to one linear map j* : V — > f)*. We can choose compatible 
system of positive roots for H and L, so that j* defines a one to one map from L 
to H. Then j*Vfc = Vfi,, for any highest weight /i G L. 

Let M be a proper Hamiltonian A"-manifold which is prequantized by a line 
bundle C. Since j : H — > L is onto we have: 

• Any A G R~°°(K) is A-admissiblc if and only if A is L-admissible, and 
E\ H =3*(E\ L ). 

• M is proper as a Hamiltonian A-manifold if and only if it is proper as a 
Hamiltonian L-manifold. 

Hence 

Q- I T{M)\ H = f{Q- i r(M)\ L ) = i*(Q-°°(M)) 

= Efi(%)v/ M , 

where M^l is the symplectic reduction at /i relative to the action of L on M. Our 
proof is then finished if we check that 

(7-48) Q(M Mli ) = Q(M,. (M) , ff ) 
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holds for any ft G L. 

The one to one map j* : I* — > f)* satisfies h ■ = j*(j(h) ■ £) for any h e H 
and £ G [*. Hence the map j* defines a 95-cquivariant symplectomorphism between 
the coadjoint orbits L ■ £ and H ■ 

Let fi G L. We work now with the proper Hamiltonian L-manifold X := M x L ■ /i 
which is prequantized by the line bundle Cx '■= £®Cr_ui. Let : M x L • — ► [* 
be the moment map relative to the L-action. Let Hl be the Hamiltonian vector 
field of the function \\ $ L || 2 . 

The "pushed" Thorn symbol Thom WL (X) is L-transversaly elliptic when whe 
restrict it to a i-invariant relatively compact open subset 1A such that 

9WnCr(|| $ L f) =0. 

Then we may consider the equivariant index Index^ (Thorn^ 1, ® C-x)- We 
know from Theorem 13. 101 that 

(7.49) [lndex£(Thom^(;r)| w ® C X )] L - Q(M Mii ) 
when ^^O) C U. 

Now we look at A" as a Hamiltonian iJ-manifold through the onto morphism 
j : H — > i: then X ~ M X H ■ Let = j* o $ L be the cooresponding 

moment map. Since j* is one to one, the functions || $l || 2 and || $h || 2 coincides 
if we choose appropriate invariant scalar products on I* and t)* . Then we have 
^(O) = $^(0) and Hl=Hh- As before Theorem EE] gives 

(7.50) [lndex^(Thom WH (^)| M ® Cx)]" = Q(M rHflhH ). 

Since [E] L = [j*E] H for any E G R~°°(L), the relations f?39j) and (f7750|) imply 
finally (|7m 

□ 
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